InfoNCE, where NCE stands for Noise-Contrastive Estimation, is a type of contrastive loss function used for self-supervised learning.

Given a set $X =${$x\_{1}, \dots, x\_{N}$} of $N$ random samples containing one positive sample from $p\left(x\_{t+k}|c\_{t}\right)$ and $N − 1$ negative samples from the 'proposal' distribution $p\left(x\_{t+k}\right)$, we optimize:

$\mathcal{L}\_{N} = - \mathbb{E}\_{X}\left[\log\frac{f\_{k}\left(x\_{t+k}, c\_{t}\right)}{\sum\_{x\_{j}\in{X}}f\_{k}\left(x\_{j}, c\_{t}\right)}\right]$

Optimizing this loss will result in $f\_{k}\left(x\_{t+k}, c\_{t}\right)$ estimating the density ratio, which is:

$f\_{k}\left(x\_{t+k}, c\_{t}\right) \propto \frac{p\left(x\_{t+k}|c\_{t}\right)}{p\left(x\_{t+k}\right)}$