StoGCN is a control variate based algorithm which allow sampling an arbitrarily small neighbor size. Presents new theoretical guarantee for the algorithms to converge to a local optimum of GCN.

**Big-Little Modules** are blocks for image models that have two branches: each of which represents a separate block from a deep model and a less deep counterpart. They were proposed as part of the [BigLittle-Net](https://paperswithcode.com/method/big-little-net) architecture. The two branches are fused with a linear combination and unit weights. These two branches are known as Big-Branch (more layers and channels at low resolutions) and Little-Branch (fewer layers and channels at high resolution).

Big-Little Module

Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition

StoGCN

Stochastic Training of Graph Convolutional Networks with Variance Reduction

GCT first collects global information by computing the l2-norm of each channel. Next, a learnable vector $ \alpha $ is applied to scale the feature. Then a competition mechanism is adopted by channel normalization to interact between channels.

Unlike previous methods, GCT first collects global information by computing the $l_{2}$-norm of each channel. 
Next, a learnable vector $\alpha$ is applied to scale the feature.
Then a competition mechanism is adopted by 
channel normalization to interact between channels. 
Like other common normalization methods, 
a learnable scale parameter $\gamma$ and bias $\beta$ are applied to 
rescale the normalization.
However, unlike previous methods,
GCT adopts tanh activation to control the attention vector.
Finally, it not only multiplies the input by the attention vector but also adds an identity connection. GCT can be written as: 
\begin{align}
    s = F_\text{gct}(X, \theta) & = \tanh (\gamma CN(\alpha \text{Norm}(X)) + \beta)
\end{align}
\begin{align}
    Y & = s  X + X
\end{align}

where $\alpha$, $\beta$ and $\gamma$ are trainable parameters. $\text{Norm}(\cdot)$ indicates the $L2$-norm of each channel. $CN$ is  channel normalization.

A GCT block has fewer parameters than an SE block, and as it is  lightweight, 
 can be added after each convolutional layer of a CNN.

Source	Stochastic Training of Graph Convolutional Networks with Variance Reduction
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: StoGCN?

Viet-Anh on Software