Conversation
Auto-attach the UI-selected command to the user message produced during an audio turn, so voice-dictated messages carry the command just like typed ones. The command is sent with audio_start, stored on the session for the turn, cleared on audio_end, and attached in Message.__post_init__. Co-Authored-By: GitHub Copilot <noreply@github.com>
There was a problem hiding this comment.
All reported issues were addressed across 7 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Message.from_dict deserializes incoming client messages and resumed thread history. Because the audio-command fallback lives in the shared __post_init__, a command-less typed message (or resumed step) created during an active audio turn would wrongly inherit the turn's command. Reset the command from the payload in from_dict so deserialized messages stay authoritative; app-constructed transcription messages still inherit as intended. Addresses PR review feedback. Co-Authored-By: GitHub Copilot <noreply@github.com>
There was a problem hiding this comment.
Pull request overview
Carries selected commands through voice/STT turns so dictated messages match typed-message behavior.
Changes:
- Sends the selected command with
audio_start. - Stores and attaches the command to generated user messages.
- Adds command inheritance and deserialization tests.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
libs/react-client/src/useChatInteract.ts |
Emits command with audio start. |
libs/react-client/src/useAudio.ts |
Forwards command when starting audio. |
frontend/src/components/chat/MessageComposer/VoiceButton.tsx |
Reads and passes the selected command. |
backend/chainlit/socket.py |
Manages command state across audio turns. |
backend/chainlit/session.py |
Adds session command state. |
backend/chainlit/message.py |
Attaches commands to generated user messages. |
backend/tests/test_message.py |
Tests command attachment behavior. |
Suppressed comments (1)
backend/chainlit/socket.py:492
- This unconditional clear races with a subsequent audio turn.
endConversationmarks the clientoffbefore emittingaudio_end, while this handler awaits the app'son_audio_end; a quick newaudio_startcan therefore overwritecurrent_command, after which the older handler both exposes the new command to the old callback and clears it here. Associate the command with a turn/context (or serialize audio turns) so overlapping handlers cannot misattribute or erase commands.
session.current_command = None
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
A declined on_audio_start (or disabled audio) previously left the UI-selected command on the session. Since the frontend sends no audio_end for a rejected start, the turn cleanup never ran and a later server-created user message could inherit the stale command. Record the command only once the connection is accepted. Addresses PR review feedback. Co-Authored-By: GitHub Copilot <noreply@github.com>
|
@codex review |
Co-Authored-By: GitHub Copilot <noreply@github.com>
There was a problem hiding this comment.
3 issues found across 8 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="backend/chainlit/message.py">
<violation number="1" location="backend/chainlit/message.py:88">
P2: When a client sends a non-string `command` in an audio or message payload, this assignment bypasses `Message.__init__`'s string normalization and stores arbitrary JSON in `Message.command`. Validate or normalize the payload command before restoring it so persistence and callbacks receive the declared string type.</violation>
</file>
<file name="backend/tests/test_socket.py">
<violation number="1" location="backend/tests/test_socket.py:637">
P3: The PR calls out "audio_start payload is optional" as a backward-compat requirement, but every new test passes `payload={"command": "search"}`. The `payload.get("command") if payload else None` branch in `socket.py` is never exercised, so an older frontend sending `audio_start` with no payload (or a payload without a `command` key) after an accepted start would silently lose the command without any test catching it. Add a test asserting `session.current_command is None` when `accepted=True` with `payload=None` and with `payload={}`.</violation>
</file>
<file name="backend/chainlit/socket.py">
<violation number="1" location="backend/chainlit/socket.py:440">
P2: Starting a new dictation before the previous `on_audio_end` finishes clears the new turn's command here. Clear only if the ending handler still owns the active audio turn, or serialize audio turns.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| # the payload so messages rebuilt here (incoming client messages, thread | ||
| # resume) never inherit an active audio turn's command that | ||
| # __post_init__ applies to command-less user messages. | ||
| message.command = _dict.get("command") |
There was a problem hiding this comment.
P2: When a client sends a non-string command in an audio or message payload, this assignment bypasses Message.__init__'s string normalization and stores arbitrary JSON in Message.command. Validate or normalize the payload command before restoring it so persistence and callbacks receive the declared string type.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/chainlit/message.py, line 88:
<comment>When a client sends a non-string `command` in an audio or message payload, this assignment bypasses `Message.__init__`'s string normalization and stores arbitrary JSON in `Message.command`. Validate or normalize the payload command before restoring it so persistence and callbacks receive the declared string type.</comment>
<file context>
@@ -73,6 +81,12 @@ def from_dict(self, _dict: StepDict):
+ # the payload so messages rebuilt here (incoming client messages, thread
+ # resume) never inherit an active audio turn's command that
+ # __post_init__ applies to command-less user messages.
+ message.command = _dict.get("command")
+ return message
</file context>
| message.command = _dict.get("command") | |
| payload_command = _dict.get("command") | |
| message.command = str(payload_command) if payload_command else None |
|
|
||
| # Only keep the UI-selected command when audio is accepted (consumed in | ||
| # Message.__post_init__), so a declined/disabled start leaves nothing stale. | ||
| session.current_command = None |
There was a problem hiding this comment.
P2: Starting a new dictation before the previous on_audio_end finishes clears the new turn's command here. Clear only if the ending handler still owns the active audio turn, or serialize audio turns.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/chainlit/socket.py, line 440:
<comment>Starting a new dictation before the previous `on_audio_end` finishes clears the new turn's command here. Clear only if the ending handler still owns the active audio turn, or serialize audio turns.</comment>
<file context>
@@ -428,16 +428,22 @@ async def window_message(sid, data):
+ # Only keep the UI-selected command when audio is accepted (consumed in
+ # Message.__post_init__), so a declined/disabled start leaves nothing stale.
+ session.current_command = None
+
if config.features.audio and config.features.audio.enabled:
</file context>
| class TestAudioStartCommand: | ||
| """audio_start only keeps the UI-selected command when audio is accepted.""" | ||
|
|
||
| async def _run(self, *, accepted, enabled=True, payload=None): |
There was a problem hiding this comment.
P3: The PR calls out "audio_start payload is optional" as a backward-compat requirement, but every new test passes payload={"command": "search"}. The payload.get("command") if payload else None branch in socket.py is never exercised, so an older frontend sending audio_start with no payload (or a payload without a command key) after an accepted start would silently lose the command without any test catching it. Add a test asserting session.current_command is None when accepted=True with payload=None and with payload={}.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/tests/test_socket.py, line 637:
<comment>The PR calls out "audio_start payload is optional" as a backward-compat requirement, but every new test passes `payload={"command": "search"}`. The `payload.get("command") if payload else None` branch in `socket.py` is never exercised, so an older frontend sending `audio_start` with no payload (or a payload without a `command` key) after an accepted start would silently lose the command without any test catching it. Add a test asserting `session.current_command is None` when `accepted=True` with `payload=None` and with `payload={}`.</comment>
<file context>
@@ -628,3 +629,43 @@ async def test_on_chat_start_not_duplicated_on_fresh_then_reconnect(
+class TestAudioStartCommand:
+ """audio_start only keeps the UI-selected command when audio is accepted."""
+
+ async def _run(self, *, accepted, enabled=True, payload=None):
+ session = Mock()
+ session.current_command = "stale"
</file context>
Problem
When a command is selected in the composer (e.g.
/search) and the user dictates their message with the voice/STT feature instead of typing it, the message is sent without the selected command. Typing the exact same message correctly includes the command.Root cause
The selected command lives in frontend state (
persistentCommandState) and is attached to the message only in the text-composer submit path (theclient_messageevent). The audio path (audio_start→audio_chunk→audio_end) never transmits the command, and the transcribed user message is created on the backend (in the app'son_audio_endhandler), so it has no knowledge of the UI selection.Fix
Carry the selected command through the audio turn and auto-attach it to the user message produced during that turn.
Frontend — send the selected command with
audio_start:VoiceButtonreadspersistentCommandStateand passesselectedCommand?.idtostartConversation.useAudio.startConversation(command?)→useChatInteract.startAudioStream(command?)→socket.emit('audio_start', { command }).Backend — store it for the turn and attach it:
BaseSessiongains acurrent_commandfield.audio_startstoressession.current_command;audio_endclears it, so it only applies to that audio turn.Message.__post_init__auto-attachescurrent_commandtouser_messages that don't already carry a command.Backward compatibility
Fully backward compatible. The
audio_startpayload is optional (old clients that emit no payload keep working), the new frontend arguments are optional, and the existingMessage.commandfield is reused. Typed messages and assistant messages are unchanged.Testing
backend/tests/test_message.py: auto-attach onuser_message, explicit command takes precedence, no-op when no command is set, and assistant messages never inherit the command.mypy,ruff,pnpm type-check, ESLint, and Prettier all pass.Out of scope
modeshas the same gap (also only attached in the text submit path). Left for a follow-up to keep this PR focused on commands.Summary by cubic
Voice/STT-dictated messages now keep the selected command like typed messages; previously they dropped it.
audio_start; the payload is optional, so old clients keep working.audio_end.Message.__post_init__auto-attaches the session command to command-lessuser_messages;from_dictresets from the payload so typed or resumed messages never inherit an active audio turn's command.Written for commit cb0f7fe. Summary will update on new commits.