Invertible multi-input multi-output coupling module. In RevBiFPN it is used as a bidirectional multi-scale feature pyramid fusion module that is invertible.

**RegNetY** is a convolutional network design space with simple, regular models with parameters: depth $d$, initial width $w\_{0} > 0$, and slope $w\_{a} > 0$, and generates a different block width $u\_{j}$ for each block $j < d$. The key restriction for the RegNet types of model is that there is a linear parameterisation of block widths (the design space only contains models with this linear structure):

$$ u\_{j} = w\_{0} + w\_{a}\cdot{j} $$

For **RegNetX** we have additional restrictions: we set $b = 1$ (the bottleneck ratio), $12 \leq d \leq 28$, and $w\_{m} \geq 2$ (the width multiplier).

For **RegNetY** we make one change, which is to include Squeeze-and-Excitation blocks.

RegNetY

Designing Network Design Spaces

RevSilo

RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network

**Affine Coupling** is a method for implementing a normalizing flow (where we stack a sequence of invertible bijective transformation functions). Affine coupling is one of these bijective transformation functions. Specifically, it is an example of a reversible transformation where the forward function, the reverse function and the log-determinant are computationally efficient. For the forward function, we split the input dimension into two parts:

$$ \mathbf{x}\_{a}, \mathbf{x}\_{b} = \text{split}\left(\mathbf{x}\right) $$

The second part stays the same $\mathbf{x}\_{b} = \mathbf{y}\_{b}$, while the first part  $\mathbf{x}\_{a}$ undergoes an affine transformation, where the parameters for this transformation are learnt using the second part $\mathbf{x}\_{b}$ being put through a neural network. Together we have:

$$ \left(\log{\mathbf{s}, \mathbf{t}}\right) = \text{NN}\left(\mathbf{x}\_{b}\right) $$

$$ \mathbf{s} = \exp\left(\log{\mathbf{s}}\right) $$

$$ \mathbf{y}\_{a} = \mathbf{s} \odot \mathbf{x}\_{a} + \mathbf{t}  $$

$$ \mathbf{y}\_{b} = \mathbf{x}\_{b} $$

$$ \mathbf{y} = \text{concat}\left(\mathbf{y}\_{a}, \mathbf{y}\_{b}\right) $$

Image: [GLOW](https://paperswithcode.com/method/glow)

Source	RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com