vietanh.dev

CBHG Explained: Convolution Banks, Highway Layers, and a Bidirectional GRU

· 3 min read · 464 words

Authors

CBHG is a sequence-processing module introduced with the original Tacotron text-to-speech model. Its name describes its three major stages:

  • CB: a bank of one-dimensional convolutions
  • H: highway-network layers
  • G: a bidirectional gated recurrent unit (GRU)

It was designed before Transformer-based speech models became dominant, but it remains a useful example of combining local, multiscale, and long-range sequence processing.

Loading diagram…

Data flow

  1. Apply several 1D convolutions in parallel, typically with kernel widths from 1 through K.
  2. Concatenate their outputs along the channel dimension.
  3. Apply stride-1 max pooling, preserving sequence length.
  4. Use projection convolutions to return to the input channel size.
  5. Add the original input as a residual connection.
  6. Pass the result through highway layers.
  7. Run a bidirectional GRU and concatenate the forward and backward states.

The convolution bank observes patterns at several temporal scales. Highway gates then control how much transformed information replaces the residual input. Finally, the bidirectional GRU collects context from both directions.

Why Tacotron used it

Character sequences contain structure at several scales: individual symbols, short character groups, syllable-like patterns, and longer linguistic context. A single convolution width does not cover all of these efficiently. The convolution bank exposes multiple receptive fields before the recurrent stage.

Tacotron used CBHG both in its encoder and as a post-processing network. In the paper's ablation, replacing the CBHG encoder with a two-layer residual GRU produced noisier attention alignments; the authors reported more mispronunciations and worse generalization to long phrases.

Important limitations

  • Every convolution branch must preserve the same sequence length before the results can be concatenated.
  • The projection output must match the input width before the residual addition.
  • A bidirectional GRU uses future context, so the complete module is not appropriate for strictly streaming synthesis.

When to use it now

CBHG remains reasonable for compact offline sequence encoders or for reproducing Tacotron-family models. For new streaming or highly parallel workloads, causal convolutions, Conformers, or Transformer variants are usually easier to scale and deploy. The important design lesson survives: mix local features at multiple resolutions before modeling global context.

Primary source