vietanh.dev

Conditional Batch Normalization: Modulating Features with Side Information

· 2 min read · 345 words

Authors

Conditional batch normalization (CBN) turns batch normalization's fixed affine parameters into functions of a conditioning input.

Ordinary batch normalization applies

ynchw=γcxnchw−μcσc2+ϵ+βc.y_{nchw} = \gamma_c \frac{x_{nchw}-\mu_c}{\sqrt{\sigma_c^2+\epsilon}} + \beta_c.

CBN replaces γc\gamma_c and βc\beta_c with values predicted from a condition zz:

ynchw=γc(z) x^nchw+βc(z).y_{nchw} = \gamma_c(z)\,\hat{x}_{nchw} + \beta_c(z).

The condition may be a class embedding, a language representation, or another modality. This gives the conditioning signal direct control over intermediate channels instead of waiting for late feature fusion.

Loading diagram…

What the original work showed

The method was introduced for visual question answering. In that work, a language embedding predicted residual changes to the frozen scale and shift parameters throughout a pretrained ResNet:

γc(z)=γc+Δγc(z),βc(z)=βc+Δβc(z).\gamma_c(z)=\gamma_c+\Delta\gamma_c(z), \qquad \beta_c(z)=\beta_c+\Delta\beta_c(z).

This allowed the question to influence visual processing from early layers. The paper's ablations found that modulating the complete visual pipeline worked better than restricting modulation to later stages.

Practical cautions

  • CBN still depends on batch statistics. Very small or non-IID batches can make those statistics noisy.
  • At inference, confirm whether running statistics or current-batch statistics are used.
  • If batch dependence is undesirable, the same conditioning idea can be applied after layer, group, or instance normalization.
  • Bound or regularize predicted scales if the conditioning network creates unstable magnitudes.

CBN is best understood as feature-wise affine modulation attached to batch normalization. The conditioning mechanism is the reusable idea; batch normalization itself is not mandatory.

Primary source