Audio: handle no-space scripts in transcript guards

[mohammadaaftabv]

Summary

• add DisfluencyWerGuardStage, using CER for scriptio-continua languages and WER otherwise
• skip whitespace-dependent repeated-word and long-word hallucination checks for no-space scripts while retaining phrase and character-rate checks
• use CER for primary/recovery agreement on no-space scripts and record the selected metric
• add dependency-free shared error-rate and scriptio-continua helpers

This is a current-main-native port of the three no-space-script behaviors from nithinraok/Curator@1e2d639946f28dfd289a0a9456480385d4480c28 (feature commit c9b4fb4e532d56978ead910675a2ef86aa62016b). A direct cherry-pick is not viable because current main already contains newer implementations of WhisperHallucinationStage and SelectBestPredictionStage.

Behavior

SCRIPTIO_CONTINUA_LANGUAGE_CODES covers Japanese, Chinese locale variants, Cantonese, Thai, Lao, Khmer, Burmese, Tibetan, and Dzongkha, matching the reference contract. Missing or space-separated languages retain the existing WER/word-heuristic behavior.

The selector keeps main's generic field naming (primary_fallback_agreement_*) while exposing configurable keys for pipelines that use the reference's omni_asr_agreement_* names.

Validation

• ruff format --check: passed
• ruff check: passed
• focused unit suite: 127 passed
• local target-versus-reference parity:
  • target: 2baa97e7903c9bbab0b126bf9ca99cb1d3453004
  • reference: 1e2d639946f28dfd289a0a9456480385d4480c28
  • 40 frozen cases: 11 guard, 11 hallucination, 13 selector, 5 composed stage-flow cases
  • Chinese, Japanese, Thai, locale/name aliases, missing-language fallbacks, English/German/Korean controls, near matches, genuine disagreements, phrase matches, high character rate, and both word heuristics
  • 40/40 independent behavioral oracle checks passed on each arm
  • zero missing rows and zero output mismatches
  • target/reference outputs were byte-identical on two isolated executions: SHA-256 22d3baa54b38baa11270ea20111c042f5f488dbf3dc9ddde542e446b1c8aa686

The parity run intentionally starts at these deterministic text-filtering stages. No ASR model run is needed: the changed contracts consume already-produced transcript/language/duration fields and do not modify Qwen inference.

Related work

Draft PR #1881 contains an older WER-only baseline. This PR applies only the requested no-space behavior to current main and supplies the missing guard without bringing over the unrelated legacy Qwen pipeline stack.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[greptile-apps]

Confidence Score: 4/5

This PR should not merge until word-based hallucination checks remain available for whitespace-delimited or mixed-script model output associated with a no-space source language.

Findings

1.  Language Gate Skips Valid Checks ▶

Summary

• Adds DisfluencyWerGuardStage with language-dependent CER/WER selection.
• Adds reusable language classification and Levenshtein error-rate helpers.
• Uses CER for cross-model agreement on configured scriptio-continua languages.
• Preserves phrase and character-rate hallucination checks while bypassing word checks for those language labels.
• Adds focused tests for language aliases, metric selection, guard behavior, and hallucination filtering.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Transcript and language metadata] --> B[Whisper hallucination filter]
    B --> C{Scriptio-continua language?}
    C -->|Yes| D[Run phrase and character-rate checks]
    C -->|No| E[Also run repeated-token and long-word checks]
    D --> F{Hallucination flagged?}
    E --> F
    F -->|Yes, primary and fallback exist| G{Agreement metric}
    G -->|Scriptio continua| H[Compute CER]
    G -->|Other or missing language| I[Compute WER]
    H --> J{Agreement meets threshold?}
    I --> J
    J -->|Yes| K[Recover primary prediction]
    J -->|No| L[Preserve hallucination flag]
    F -->|No| M[Keep normal prediction]
    N[Original and disfluency-cleaned text] --> O{Scriptio-continua language?}
    O -->|Yes| P[Compute CER]
    O -->|No| Q[Compute WER]
    P --> R{Rate exceeds guard threshold?}
    Q --> R
    R -->|Yes| S[Restore original transcript]
    R -->|No| T[Keep cleaned transcript]

Loading

_{Reviews (1) · Last reviewed commit: "feat(audio): support no-space scripts in..."}

[greptile-apps]

Language Gate Skips Valid Checks

When a row is labeled with a scriptio-continua language, this code disables both word-based checks regardless of the actual transcript. A Japanese- or Chinese-labeled recording can still produce a whitespace-delimited hallucination such as repeated English tokens. In that case, the repeated-word and long-word checks remain applicable but never run, allowing the hallucinated transcript to pass when it does not match a configured phrase or exceed the character-rate threshold. Restrict the exemption to transcripts that actually exhibit no-space-script characteristics rather than relying only on the source-language label.