**Feature Matching** is a regularizing objective for a generator in [generative adversarial networks](https://paperswithcode.com/methods/category/generative-adversarial-networks) that prevents it from overtraining on the current discriminator. Instead of directly maximizing the output of the discriminator, the new objective requires the generator to generate data that matches the statistics of the real data, where we use the discriminator only to specify the statistics that we think are worth matching. Specifically, we train the generator to match the expected value of the features on an intermediate layer of the discriminator. This is a natural choice of statistics for the generator to match, since by training the discriminator we ask it to find those features that are most discriminative of real data versus data generated by the current model.

Letting $\mathbf{f}\left(\mathbf{x}\right)$ denote activations on an intermediate layer of the discriminator, our new objective for the generator is defined as: $ ||\mathbb{E}\_{x\sim p\_{data} } \mathbf{f}\left(\mathbf{x}\right) − \mathbb{E}\_{\mathbf{z}∼p\_{\mathbf{z}}\left(\mathbf{z}\right)}\mathbf{f}\left(G\left(\mathbf{z}\right)\right)||^{2}\_{2} $. The discriminator, and hence
$\mathbf{f}\left(\mathbf{x}\right)$, are trained as with vanilla GANs. As with regular [GAN](https://paperswithcode.com/method/gan) training, the objective has a fixed point where G exactly matches the distribution of training data.

GAN Feature Matching

Improved Techniques for Training GANs

Given a training set drawn from an unknown $d$-variate probability distribution, QuantTree constructs a histogram by recursively splitting $\mathbb{R}^d$. The splits are defined by a stochastic process so that each bin contains a certain proportion of the training set. These histograms can be used to define test statistics (e.g., the Pearson statistic) to tell whether a batch of data is drawn from $\phi_0$ or not. The most crucial property of QuantTree is that the distribution of any statistic based on QuantTree histograms is independent of $\phi_0$, thus enabling nonparametric statistical testing.

QuantTree

QuantTree: Histograms for Change Detection in Multivariate Data Streams

**GPT-4** is a transformer based model pre-trained to predict the next token in a document.

GPT-4

GPT-4 Technical Report

**MADDPG**, or **Multi-agent DDPG**, extends [DDPG](https://paperswithcode.com/method/ddpg) into a multi-agent policy gradient algorithm where decentralized agents learn a centralized critic based on the observations and actions of all agents. It leads to learned policies that only use local information (i.e. their own observations) at execution time, does not assume a differentiable model of the environment dynamics or any particular structure on the communication method between agents, and is applicable not only to cooperative interaction but to competitive or mixed interaction involving both physical and communicative behavior. The critic is augmented with extra information about the policies of other agents, while the actor only has access to local information. After training is completed, only the local actors are used at execution phase, acting in a decentralized manner.

MADDPG

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

**HDCGAN**, or **High-resolution Deep Convolutional Generative Adversarial Networks**, is a [DCGAN](https://paperswithcode.com/method/dcgan) based architecture that achieves high-resolution image generation through the proper use of [SELU](https://paperswithcode.com/method/selu) activations. Glasses, a mechanism to arbitrarily improve the final [GAN](https://paperswithcode.com/method/gan) generated results by enlarging the input size by a telescope ζ is also set forth. 

A video showing the training procedure on CelebA-hq can be found [here](https://youtu.be/1XZB87W0SaY).

HDCGAN

High-Resolution Deep Convolutional Generative Adversarial Networks

**R-CNN**, or **Regions with CNN Features**, is an object detection model that uses high-capacity CNNs to bottom-up region proposals in order to localize and segment objects. It uses [selective search](https://paperswithcode.com/method/selective-search) to identify a number of bounding-box object region candidates (“regions of interest”), and then extracts features from each region independently for classification.

R-CNN

Rich feature hierarchies for accurate object detection and semantic segmentation

Adaptive Early-Learning Correction for Segmentation from Noisy Annotations

ADELE

**Chained-Tracker**, or **CTracker**,  is an online model for multiple-object tracking. It chains paired bounding boxes regression results estimated from overlapping nodes, of which each node covers two adjacent frames. The paired regression is made attentive by object-attention (brought by a detection module) and identity-attention (ensured by an ID verification module).

The joint attention module guides the paired boxes regression branch to focus on informative spatial regions with two other branches. One is the object classification branch, which predicts the confidence scores for the first box in the detected box pairs, and such scores are used to guide the regression branch to focus on the foreground regions. The other one is the ID verification branch whose prediction facilitates the regression branch to focus on regions corresponding to the same target. Finally, the bounding box pairs are filtered according to the classification confidence. Then, the generated box pairs belonging to the adjacent frame pairs could be associated using simple methods like IoU (Intersection over Union) matching according to their boxes in the common frame. In this way, the tracking process could be achieved by chaining all the adjacent frame pairs (i.e. chain nodes) sequentially.

CTracker

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

A **Gated Convolutional Network** is a type of language model that combines convolutional networks with a gating mechanism. Zero padding is used to ensure future context can not be seen. Gated convolutional layers can be stacked on top of other hierarchically. Model predictions are then obtained with an [adaptive softmax](https://paperswithcode.com/method/adaptive-softmax) layer.

Gated Convolution Network

Language Modeling with Gated Convolutional Networks

**BigGAN-deep** is a deeper version (4x) of [BigGAN](https://paperswithcode.com/method/biggan).  The main difference is a slightly differently designed [residual block](https://paperswithcode.com/method/residual-block). Here the $z$ vector is concatenated with the conditional vector without splitting it into chunks.  It is also based on residual blocks with bottlenecks. BigGAN-deep uses a different strategy than BigGAN aimed at preserving identity throughout the skip connections. In G, where the number of channels needs to be reduced, BigGAN-deep simply retains the first group of channels and drop the rest to produce the required number of channels. In D, where the number of channels should be increased, BigGAN-deep passes the input channels unperturbed, and concatenates them with the remaining channels produced by a 1 × 1 [convolution](https://paperswithcode.com/method/convolution). As far as the
network configuration is concerned, the discriminator is an exact reflection of the generator. 

There are two blocks at each resolution (BigGAN uses one), and as a result BigGAN-deep is four times
deeper than BigGAN. Despite their increased depth, the BigGAN-deep models have significantly
fewer parameters mainly due to the bottleneck structure of their residual blocks.

BigGAN-deep

Large Scale GAN Training for High Fidelity Natural Image Synthesis

SETSe is a deterministic physics based graph embedding algorithm. It embeds weighted feature rich networks. It treats each edge as a spring and each node as a bead whose movement is constrained by the graph adjacency matrix so that the nodes move in parallel planes enforcing a minimum distance between neighboring nodes. The node features act as forces moving the nodes up and down. The network converges to the embedded state when the force produced by each node is equal and opposite to the sum of the forces exerted by its edges, creating a net force of 0.

SETSe has no conventional loss function and does not attempt to place similar nodes close to each other.

SETSe

**Normalizing Flows** are a method for constructing complex distributions by transforming a
probability density through a series of invertible mappings. By repeatedly applying the rule for change of variables, the initial density ‘flows’ through the sequence of invertible mappings. At the end of this sequence we obtain a valid probability distribution and hence this type of flow is referred to as a normalizing flow.

In the case of finite flows, the basic rule for the transformation of densities considers an invertible, smooth mapping $f : \mathbb{R}^{d} \rightarrow \mathbb{R}^{d}$ with inverse $f^{-1} = g$, i.e. the composition $g \cdot f\left(z\right) = z$. If we use this mapping to transform a random variable $z$ with distribution $q\left(z\right)$, the resulting random variable $z' = f\left(z\right)$ has a distribution:

$$ q\left(\mathbf{z}'\right) = q\left(\mathbf{z}\right)\bigl\vert{\text{det}}\frac{\delta{f}^{-1}}{\delta{\mathbf{z'}}}\bigr\vert = q\left(\mathbf{z}\right)\bigl\vert{\text{det}}\frac{\delta{f}}{\delta{\mathbf{z}}}\bigr\vert ^{-1} $$

where the last equality can be seen by applying the chain rule (inverse function theorem) and is a property of Jacobians of invertible functions. We can construct arbitrarily complex densities by composing several simple maps and successively applying the above equation. The density $q\_{K}\left(\mathbf{z}\right)$ obtained by successively transforming a random variable $z\_{0}$ with distribution $q\_{0}$ through a chain of $K$ transformations $f\_{k}$ is:

$$ z\_{K} = f\_{K} \cdot \dots \cdot f\_{2} \cdot f\_{1}\left(z\_{0}\right) $$

$$ \ln{q}\_{K}\left(z\_{K}\right) = \ln{q}\_{0}\left(z\_{0}\right) − \sum^{K}\_{k=1}\ln\vert\det\frac{\delta{f\_{k}}}{\delta{\mathbf{z\_{k-1}}}}\vert $$

The path traversed by the random variables $z\_{k} = f\_{k}\left(z\_{k-1}\right)$ with initial distribution $q\_{0}\left(z\_{0}\right)$ is called the flow and the path formed by the successive distributions $q\_{k}$ is a normalizing flow.

Normalizing Flows

Variational Inference with Normalizing Flows

Forward gradients are unbiased estimators of the gradient $\nabla f(\theta)$ for a function $f: \mathbb{R}^n \rightarrow \mathbb{R}$, given by $g(\theta) = \langle \nabla f(\theta) , v \rangle v$. 

Here $v = (v_1, \ldots, v_n)$ is a random vector, which must satisfy the following conditions in order for $g(\theta)$ to be an unbiased estimator of $\nabla f(\theta)$

* $v_i \perp v_j$ for all $i \neq j$
* $\mathbb{E}[v_i] = 0$ for all $i$
* $\mathbb{V}[v_i] = 1$ for all $i$

Forward gradients can be computed with a single jvp (Jacobian Vector Product), which enables the use of the forward mode of autodifferentiation instead of the usual reverse mode, which has worse computational characteristics.

Forward gradient

Gradients without Backpropagation

The **GAN Hinge Loss** is a hinge loss based loss function for [generative adversarial networks](https://paperswithcode.com/methods/category/generative-adversarial-networks):

$$ L\_{D} = -\mathbb{E}\_{\left(x, y\right)\sim{p}\_{data}}\left[\min\left(0, -1 + D\left(x, y\right)\right)\right] -\mathbb{E}\_{z\sim{p\_{z}}, y\sim{p\_{data}}}\left[\min\left(0, -1 - D\left(G\left(z\right), y\right)\right)\right] $$

$$ L\_{G} = -\mathbb{E}\_{z\sim{p\_{z}}, y\sim{p\_{data}}}D\left(G\left(z\right), y\right) $$

GAN Hinge Loss

Geometric GAN

**CoOp**, or **Context Optimization**, is an automated prompt engineering method that avoids manual prompt tuning by modeling context words with continuous vectors that are end-to-end learned from data. The context could be shared among all classes or designed to be class-specific. During training, we simply minimize the prediction error using the cross-entropy loss with respect to the learnable context vectors, while keeping the pre-trained parameters fixed. The gradients can be back-propagated all the way through the text encoder, distilling the rich knowledge encoded in the parameters for learning task-relevant context.

CoOp

Learning to Prompt for Vision-Language Models

**State-Aware Tracker** is a pipeline for semi-supervised video object segmentation. It takes each target object as a tracklet, which not only makes the pipeline more efficient but also filters distractors to facilitate target modeling. For more stable and robust performance over video sequences, SAT gets awareness for each state and makes self-adaptation via two feedback loops. One loop assists SAT in generating more stable tracklets. The other loop helps to construct a more robust and holistic target representation.

State-Aware Tracker

State-Aware Tracker for Real-Time Video Object Segmentation

**K-Net** is a framework for unified semantic and instance segmentation that segments both instances and semantic categories consistently by a group of learnable kernels, where each kernel is responsible for generating a mask for either a potential instance or a stuff class. It begins with a set of kernels that are randomly initialized, and learns the kernels in accordance to the segmentation targets at hand, namely, semantic kernels for semantic categories and instance kernels for instance identities. A simple combination of semantic kernels and instance kernels allows panoptic segmentation naturally. In the forward pass, the kernels perform [convolution](https://paperswithcode.com/method/convolution) on the image features to obtain the corresponding segmentation predictions.

K-Net is formulated so that it dynamically updates the kernels to make them conditional to their activations on the image. Such a content-aware mechanism is crucial to ensure that each kernel, especially an instance kernel, responds accurately to varying objects in an image. Through applying this adaptive kernel update strategy iteratively, K-Net significantly improves the discriminative ability of the kernels and boosts the final segmentation performance. It is noteworthy that this strategy universally applies to kernels for all the segmentation tasks.

It also utilises a bipartite matching strategy to assign learning targets for each kernel. This training approach is advantageous to conventional training strategies as it builds a one-to-one mapping between kernels and instances in an image. It thus resolves the problem of dealing with a varying number of instances in an image. In addition, it is purely mask-driven without involving boxes. Hence, K-Net is naturally [NMS](https://paperswithcode.com/method/non-maximum-suppression)-free and box-free, which is appealing to real-time applications.

K-Net

K-Net: Towards Unified Image Segmentation

Training a denoiser on signals gives you a powerful prior over this signal that you can then use to sample examples of this signal.

AI Glossary

GAN Feature Matching

QuantTree histograms

GPT-4

MADDPG

High-resolution Deep Convolutional Generative Adversarial Networks

R-CNN

Adaptive Early-Learning Correction

Chained-Tracker