**Local SGD** is a distributed training technique that runs [SGD](https://paperswithcode.com/method/sgd) independently in parallel on different workers and averages the sequences only once in a while.

**VGG Loss** is a type of content loss introduced in the [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://paperswithcode.com/paper/perceptual-losses-for-real-time-style) super-resolution and style transfer framework. It is an alternative to pixel-wise losses; VGG Loss attempts to be closer to perceptual similarity. The [VGG](https://paperswithcode.com/method/vgg) loss is based on the [ReLU](https://paperswithcode.com/method/relu) activation layers of the pre-trained 19 layer VGG network. With $\phi\_{i,j}$ we indicate the feature map obtained by the $j$-th [convolution](https://paperswithcode.com/method/convolution) (after activation) before the $i$-th maxpooling layer within the VGG19 network, which we consider given. We then define the VGG loss as the euclidean distance between the feature representations of a reconstructed image $G\_{\theta\_{G}}\left(I^{LR}\right)$ and the reference image $I^{HR}$:

$$ l\_{VGG/i.j} = \frac{1}{W\_{i,j}H\_{i,j}}\sum\_{x=1}^{W\_{i,j}}\sum\_{y=1}^{H\_{i,j}}\left(\phi\_{i,j}\left(I^{HR}\right)\_{x, y} - \phi\_{i,j}\left(G\_{\theta\_{G}}\left(I^{LR}\right)\right)\_{x, y}\right)^{2}$$ 

Here $W\_{i,j}$ and $H\_{i,j}$ describe the dimensions of the respective feature maps within the VGG network.

VGG Loss

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Local SGD

Local SGD Converges Fast and Communicates Little

**Attention Mesh** is a neural network architecture for 3D face mesh prediction that uses attention to semantically meaningful regions. Specifically region-specific heads are employed that transform the feature maps with spatial transformers.

Source	Local SGD Converges Fast and Communicates Little
Year	2000
Data Source	Papers with Code archive (CC BY-SA)