TeNC: Low Bit-Rate Speech Coding with VQ-VAE and GAN

Abstract

Speech coding aims at compressing digital speech signals with fewer bits and reconstructing it back to raw signals. But conventional codecs usually need a high bit-rate to achieve reconstructed speech of high quality. In this paper, we propose an end-to-end generative codec with a VQ-VAE based auto-encoder and the generative adversarial network (GAN), which achieves high-fidelity reconstructed speech at a low bit-rate about 2 kb/s. The compression process of speech coding is carried out by a down-sampling module of the encoder and a learnable discrete codebook. Our experiments confirm the effectiveness of the proposed model in both objective and subjective tests, which significantly outperforms the conventional codecs at low bit-rate in terms of speech quality and speaker similarity.

( TeNC: Proposed NN Codec with VQ-VAE and GAN. Opus works in CBR mode with 10ms frame size.)

Item 1

Text: I never see lou's scythe over here.

Item 2

Text: On either side of the road for a mile before you reached the foot of the hill stood tall osage orange hedges, their glossy green marking off the yellow fields.

Item 3

Text: Well now ennis i declare you have a head and so has my stick.

Item 4

Text: A great saint saint francis xavier.

Item 5

Text: He had the faith in him that moves mountains.

Item 6

Text: On friday confession will be heard all the afternoon after beads.

Item 7

Text: The first of our vague but indubitable data is that there is knowledge of the past.

Item 8

Text: This knowledge is memory in one sense though in another it is not.

Item 9

Text: Thus no knowledge as to the past is to be derived from the feeling of familiarity alone.

Item 10

Text: He started at the thought he hurried forth sadly.

Item 11

Text: She had been born within its borders within its borders she had lived and grown and within its borders she had met her love.

Item 12

Text: The lagoon had been level with the dykes a week ago and now.