pl: mark the unicode translations as verified (t -> T) - #741
pl: mark the unicode translations as verified (t -> T)#741michaldziwisz wants to merge 3 commits into
Conversation
|
There are conflicts in unicode-full.yaml that appear to be becasue of the t->T change. Can you resolve them. I don't accidentally want to break something |
Mechanical change: 2435 text keys raised from t/ot/ct to T/OT/CT in the two unicode files. No text is touched. The Polish rules were written before the lowercase/uppercase convention was in use, so entries that have been translated all along still carried the "needs review" key. The audit tool therefore reported them as untranslated, which made the real gaps impossible to see: audit-translations pl, untranslated text: 3162 -> 727 The remaining 727 are in the rule files, where the entries need reading rather than a mechanical pass, so they are deliberately left alone. Every raised entry was checked to differ from the English source, i.e. it really is translated. Entries whose text legitimately equals English (Roman numerals, ligatures, proper names, whitespace) were handled separately in the previous PR; entries still holding English text are NOT raised, since that would assert a translation that does not exist. Verification: cargo test --test languages Languages::pl 609 passed, 0 failed diff: 2436 insertions, 2436 deletions, zero changes outside key case The last point is checked by normalising the key case in the diff and confirming no unpaired lines remain (without normalising, 4710 remain, so the check does discriminate).
5f1a8dd to
7b5f401
Compare
|
Done - rebased onto current One clarification on the cause, because it matters for "I don't accidentally want to break something": the conflicts were not the t->T change disagreeing with anything in Measurements, so the safety claim is checkable rather than asserted:
To be precise about what is not fixed here, since the numbers do not go to zero: the remainder in those two files is small keys inside the rule blocks at the top of each file (letter/greek/capital handling, e.g. One note for the release process: |
The previous commit raised the key on entries written on a single line
(`- "±": [t: "..."]`), but not on entries whose body is a block:
- "°":
- test:
if: "parent::*[name(.)='msup' ...]"
then: [t: "stopni"]
else: [t: "stopień"]
Those are exactly the entries carrying inflection or a TTS condition, so
312 of the most deliberate translations in the file were left looking
unreviewed. This raises them: 64 in unicode.yaml, 247 in unicode-full.yaml
and one single-line entry (U+23E6) missed earlier.
One entry is deliberately left with a lower-case key: `t: ""` for U+00A0,
the else-branch of the `@data-empty-in-2D` test. That is silence rather
than text, and an upper-case key means "a translator checked this
wording" - there is no wording to check. English keeps a lower-case key
there too.
No content changes; only key case. `cargo test --test languages
Languages::pl` gives 609 passed, 0 failed, which is the same result as
before this commit, as expected for a change the engine treats as
equivalent (src/speech.rs matches "t"|"T" in one match arm).
Two loose ends in the Polish localization.
**Rule-file keys.** 248 entries in the rule files still used a lower-case
key while carrying translated Polish text. Content is untouched - every
one of the 248 changed lines differs from its predecessor only in the
case of the key name, verified line by line after normalizing the key.
17 entries are deliberately left lower-case: five are the letters C, N,
Q, R, Z in `dimension-number-sets` (the letter itself is the wording, as
in English); `ct: "s"`/`ct: "-"` in the unit rule are the plural suffix
and prefix separator; `navigate.yaml:176` is a Polish euphony variant
("ze" before "st-"); the remaining seven are XPath expressions rather
than speech.
**Tests for the number-set rules.** These rules had no Polish test at
all, in a file where two silent defects were already found. New
`tests/Languages/pl/number_sets.rs` covers the bare symbols, the
dimension form, both sign variants, and ClearSpeak agreeing with
SimpleSpeak.
The `number_set_with_variable_exponent` case is the negative control:
`dimension-number-sets` fires on `count(*)=2`, so a non-numeric exponent
must fall through to the full phrase ("liczby całkowite do potęgi n")
rather than being read as a dimension. Without it, a rule that swallowed
every superscript would still pass.
Polish keeps the letter with a dimension ("R 3"), matching en, de, fr,
nb, sv, fi and es; Hungarian is the only language that translates in that
position too. Measured with the SAPI voice NVDA actually uses: "R 3" is
byte-identical to a phonetic "er 3" and "N 4" to "en 4", so the letters
are pronounced as their Polish names.
cargo test --test languages Languages::pl 614 passed, 0 failed
What this does
Raises the "needs review" key (
t/ot/ct→T/OT/CT) on Polish entries that have in fact been translated all along, and adds tests for one rule that had none. No wording is changed anywhere in this PR.Three commits, reviewable separately:
5f1a8ddaunicode.yaml/unicode-full.yaml845be3dfb4e8ca92ClearSpeak_Rules,navigate,SharedRules/*,overview) + new testsWhy
The Polish rules were written before the lower/upper-case convention was in use, so translated entries still carried the "needs review" key. The audit tool counted every one of them as untranslated:
The 3162 made the real gaps invisible. Working out that only ~80 unicode entries were genuinely untranslated (handled in #738) required a separate tool comparing the text against the English source, because key case alone says nothing about whether the work was done.
Why the second commit exists
The tool I used for the first pass only recognised entries written on one line (
- "±": [t: "..."]). Entries whose body is a block were invisible to it:Those are exactly the entries carrying inflection or a TTS condition — 312 of the most deliberate translations in the file were left looking unreviewed. Fixing the tool and re-running it over my own earlier work is what surfaced them.
What is not raised
Only entries whose text differs from the English source are touched — that is the evidence a translation exists. 18 entries deliberately keep a lower-case key:
t: ""for U+00A0 (the@data-empty-in-2Delse-branch). That is silence, not text; an upper-case key asserts "a translator checked this wording" and there is no wording to check. English keeps a lower-case key there too.C,N,Q,R,Zindimension-number-sets— the letter is the wording, as in English.ct: "s"andct: "-"in the unit rule: the plural suffix and the prefix separator, not words. (The"s"is an English plural ending on Polish unit names; pl: inflect unit names according to the numeral, not with an English "s" #744 removes it.)navigate.yaml:176, a Polish euphony variant ("ze" before "st-"), which happens to equal a Russian letter name in the English corpus.Entries whose text legitimately equals English (Roman numerals, ligatures, proper names, Greek and Cyrillic letter names) were reviewed individually in #738.
Tests for the number-set rules
ℂ ℕ ℚ ℝ ℤhad no Polish test at all, in a file where two silent defects have already been found. Newtests/Languages/pl/number_sets.rscovers the bare symbols, the dimension form, both sign variants, and ClearSpeak agreeing with SimpleSpeak.number_set_with_variable_exponentis the negative control:dimension-number-setsfires oncount(*)=2, so a non-numeric exponent must fall through to the full phrase ("liczby całkowite do potęgi n") instead of being read as a dimension. Without it, a rule that swallowed every superscript would still pass.Polish keeps the letter with a dimension ("R 3"), matching en, de, fr, nb, sv, fi and es; Hungarian is the only language that translates in that position too. Checked against the SAPI voice NVDA actually uses: "R 3" renders byte-identical to a phonetic "er 3", and "N 4" to "en 4", so the letters really are spoken as their Polish names.
Verification
Zero changes outside key case. Each removed line was paired with its added line and compared after normalising the key name; all 248 rule-file differences are
t -> Twith content, comments and indentation identical character for character. For the unicode pass the same check was done with:Without the normalisation 4710 unpaired lines remain, so the check discriminates rather than trivially passing.
Kept separate from #738 on purpose: 2995 lines of key case should not bury five reviewable fixes.