feat(voicing): add mu-law domain for voicing/energy extraction - #333
Open
KakaruHayate wants to merge 1 commit into
Open
KakaruHayate wants to merge 1 commit into
KakaruHayate wants to merge 1 commit into
Conversation
Introduce 'mulaw' voicing_domain and mu-law energy handling across configs and preprocessing. Configs updated to include voicing_domain='mulaw' (acoustic/variance templates and instances). Acoustic/variance binarizers now pass hparams['voicing_domain'] to get_voicing. Parameter adaptor clamps voicing max to 0 when domain is 'mulaw'. get_energy_librosa and get_voicing gain new domain/mu params and implement mu-law compression (default mu=255) to produce a dB-like range compatible with existing pipelines. These changes add support for mu-law representation of voicing/energy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds a
voicing_domainconfiguration key that selects how the voicing curve (and the RMS energy it is derived from) is represented:'db'(default): the current behaviour - raw RMS of the harmonic part converted to decibels.'mulaw': mu-law compression (mu = 255) of the same RMS values, rescaled to[-96, 0]so it stays compatible with the dB-like range the rest of the pipeline expects. The compression gives low-energy frames a larger share of the available range, which improves resolution on unvoiced and weakly voiced segments.Implementation:
utils/binarizer_utils.get_energy_librosaandget_voicinggaindomainandmuparameters.hparams['voicing_domain']through when extracting voicing.0when the domain is'mulaw';voicing_db_minstill applies as the lower bound.docs/ConfigurationSchemas.mdgains avoicing_domainentry.Configuration
voicing_domainis read from the acoustic and variance configs and falls back to'db'when unset, so configurations that do not set it keep exactly the current behaviour.The shipped
configs/acoustic.yaml,configs/variance.yamland the two template configs do setvoicing_domain: 'mulaw'. Let me know if you would rather keep the shipped recipes on'db'and treat mulaw as strictly opt-in - I am happy to drop those four lines.Compatibility
Switching the domain changes both the stored voicing curves and the normalization range of the voicing branch, so it requires re-running binarization and training from scratch to take full effect. This is noted in the new schema entry.
Notes
Supersedes #304, which was closed after unrelated work ended up on the same branch. This PR contains only the mu-law change, rebased onto the current
main.